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ABSTRACT 

Designed to aid educators in organizing and 
conducting evaluation programs for local vocational and technical 
education, this pamphlet discusses ten steps on guidelines to be used 
in the procedures, in the order of their use, they are: (1) 
contacting the state vocational education department, (2) deciding 
how much to evaluate, (3) selecting the evaluating team, (4) deciding 
what to evaluate, (5) orientating the evaluation team, (6) providing 
materials for the evaluation t:eam, (7) collecting and recording 
information, (8) reporting the results, (9) utilizing the results, 
and (10) writing tfie followup report to the evaluation team. Selected 
references provide a more in-depth treatment of the subject. 
(Author/SN) 
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The Center tor Vocational and Technical Education is an independent 
unit on The Oiiio State University caapus. It serves a catalytic role 
in establishing consortia to focus on relevant problems in vocational 
and technical education. The Center is cosrprehensive in its coomitnent 
and responsibilily, nultidisciplinary in its approach and interinstitu- 
tional in its program. 

The Center's mission is to strengthen the capacity of state educa- 
tional systems to provide effective occupational education programs con- 
sistent vith individual needs and manpower requijremrats by: 

. Conducting research and developaent to fill voids in existing 
knovledge azid to develop methods for applying knowledge. 

* Brogranatic focus on state leadership development, vocational 
teacher education, curriculimi, vocational choice and adjustment. 

* Stimulating and strengthening the capacity of other agencies and 
institutions to create durable solutions to significant problems. 

* Providing a national infomation storage, retrieval and dissemi- 
nation system for vocational and technical education through the 
affiliated ERIC Clear ilig^ouse. 



The material in thi3 publication was prepared pursuant to a con- 
tzMt with the National Institute of Education, U.S. Department of 
Health, Education and Welfare. Contaractors undertaking such projects 
under government sponsorship are encouraged to express fireely their 
judgnent in professional and technical matters. R>ints of view or opin- 
ions do not, therefore, necessarily represent official national Institute 
of Education position or policy. 



FOREWORD 



As the demand for accoxmtability in education has increased , 
educators have searched for improved means by vhich to evaluate their 
programs. This publication is designed to aid educators in organizing 
and conducting evaluation programs for vocational and technical edu- 
cation. The author identifies ten steps or guidelines to serve as a 
skeletstl structtire in determining Cfvaluation procedures , evaluation 
committee organization, and the collection and utilization of data. 
An accompanying bibliography provides a list of original sources for 
the reader wishing a more in- depth treatment of the 8ub;)ect. 

The profession is iiidebted to Richard Edsall for his scholarship 
in the preparation of his report. Recognition is also due John Klit, 
Illinois Division of Vocational-Technical Education; and Ronald W. 
Stadt, Southern Illinois University; for their critical review of the 
manxiscript prior to final revision and publication. Wesley E. Budke, 
Assistant Director for Information Utilization at The Center, coordi- 
nated the publication and development and Alice J. Brown provided the 
technical editing. 

Robert E. Taylor 
Director 

The Center for Vocational and 
Technical Education 

ERIC Clearinghouse on Vocational 
and Technical Education 



INTRODUCTION 



This pamphlet ic designed to assist local superintendents, princi- 
pals, vocational directors, program supervisors, and teachers in organiz- 
ing and conducting an evaluation of a local vocational program by utiliz- 
ing a specifically selected evaluation committee. This evaluation ccam- 
mittee is assembled, conducts the evaluation in a short period of time, 
reports their findings and recommendations and is disbanded. The organi- 
zation and use of tbis comndttee complements the evaluation efforts of 
teachers, administrators, and local program advisory committees and 
provides additional management information for \ise in updating and 
improving the vocational program. 

The guidelines in this pamphlet are designed to serve as a founda- 
tion to which local administrators and teachers can add their own ideas 
and innovations to obtain the greatest benefits from the efforts of the 
evaluation team. 
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STEP ONE 

Contact Your State Vocational Education Department 



For efficiency and effectiveness in conducting this effort, the 
local administrator should contact the evaluation section of his state 
department of vocational education. Local and state evaluation efforts 
must be complementary. Data gathered during the process should be of 
benefit to all levels of the education system for efficient management 
and decision-making. Also, special expertise, technical assistance, 
detailed guides and procedures are often available in the state depart- 
ment for use by the local school officials. 

If there is no evaluation section within the state department, the 
program operations, program supervision, or similar sections can often 
provide some assistamce with the evaluation. 



Person to contact in state department: 



Name 



Address 



City State Zip 



Telephone 



STEP TWO 
How Much to Evaluate 



The decision about how much of the local program offering to evalu- 
ate should be made by the local administrator in consultation with his 
staff. If many different programs are offered, consideration must be 
given to the amount of time etnd resources that can be given to the evalu 
ation effort. Assessing only a few of the programs gives the advantage 
of having transfer effects to programs not evaluated. Evaluating all 
programs offers the opportunity for comparison. 

Evaluating a few different programs annually and, at selected inter 
vals (perhaps five years) evaluating all programs during the same year, 
is one suggested method. 



Program(s) to be evaluated: 



STEP THREE 
Selecting the Evaluation Team 



Bringing in a special evaluation team can add a new perspective to 
the program review, and new ideas for program improvement can result. 
An evaluation team specifically selected for their knowledge and exper- 
tise in one area can give an in-depth perspective to that program. A 
group with expertise in several areas should be selected if only one 
committee is to evaluate several programs or an entire school offering. 

The number of persons to be selected on the committee will vary 
with the scope of the task. Three to five persons are probably suffi- 
cient for a single program. Ten to fifteen persons on a team should be 
sufficient to evaluate most total school offerings.- The selection of 
evaluation committee members is very important. Byram (1965) includes 
the following in sane suggested criteria for selection: 

Knowledge of the program area - including skill requirements 
and employment needs. 

Interest in the program. 

Judgnent ability. 

Communication ability. 

Willingness to serve. 

Potential sources for team members include: 

1) Etoployers (or potential employers) of students from the program. 

2) Former students from the program. 

3) Parents of students in the program. 

k) Vocational teachers from the same or similar programs in other 
schools. 

5) Representatives fran the teacher training staff for that pro- 
gram . 

6) Representatives of the state supervisory staff. 

7) Students in the program . 
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Program to be evcduated: 



Agreed 

to Serve Phone No. Persons Selected Address 
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STEP FOUR 
What to Evaluate 



The eveduation team can review as many different aspects of the 
selected program(s) as time and resources allow. The specific areas to 
be examined should be selected by the local administrator, or by the 
evaluation team in consultation with the administrative staff. 

PROCatAM EIEMEMTS SELECTED FOR BEVIEW 

Purposes and Objectives 
Adminif ration 

Facilities 

Curriculum (Materieas) 

Instruction (Methods) 

Student Activities 

(Youth Organizations) 

Advisory Committees 

Community Relations 

Staff (Abilities and Qualifications) 

Guidance, Placement auid Follow-Up ^ 

Program Evaluation 



STEP FIVE 
Orientation of Evaluation Team 



After the team meoibers have been contacted and have agreed to serve, 
the local administrator should set a convenient meeting time and piece. 
At this meeting^ an orientation is essential. The local administ^tor 
should present the purpose of the evaluation and the duties and respon- 
sibilities of the members. Suggested itans for the orientation are: 

1) Purposes of the evaluation, including: 

a. Oetemining strengths and weaknesses of the program, 
b Assisting the teacher in appraising the effectiveness of 
his activities. 

c. Providing school officials with decision-making informa- 
tion necessary fot prograa. jnprovement . 

2) Ibilosoidiy of the school and the program. 

3) Admission req.uirements. 

If) Review of program objectives stated in terms of: 

a. The learner. 

b. The change in behavior desired. 

c. Areas of concern relative to each objective. 

3) Specific elements of the program to be evaluated. 

6) Collection of data and other information needed for assessment. 

7) Specific organization for the evaluation Including: 

a. Organization of the ccnnittee - appointment of chairman* 

b. Appointment of siibcoamittees . 

c. Selection of times and places for interim meetings. 

d. Arrangements for class visitations. 

e. Arrangements for interviews with teachers and students. 

8) Procedures for reporting including: 

a. Developnent of conclusions. 

b. Development of recommendations. 

c. Production of the final report. 



STEP SIX 

Provide Materials for the Evaluation Team 



Make arrangements for comfortable and convenient meeting rooms. 
Work materials incliading instruments for data collection , guides for 
reviewing selected items, paper, and pencils sboiild be provided. A 
secretary should be assigned to type necessary reports. A technical 
writer should be assigned to assist in preparing the final report. 



Secretary 



Ifriter 



RtODber 

Supply Items Needed 
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STEP SEVEN 
Information Collection and Recording 



The collection of evaluative information shoiild focus on quantita- 
tive and qualitative measures relating to the program objectives. On- 
phasis shotild be given to items or facts \diich will aid in improving the 
quality of the program* 

Many evaluation instrunents are available. It is not the pusrpose 
of this guide to produce yet another one. While the quality of the 
available instruments varies > none have value if they are not utilized. 
With any instrument > evaluation team m^nbers must be given maximum flexi 
bility and encouraged to utilize tiieir own expertise in assessing the 
program. Conprehensive forms, unless utilized carefully, can over- 
structure the evaluation process. 

State vocational education departments often have instruments avail 
able for use by local agencies. Step One in this guide serves the pur- 
pose of secxuring the instruments and/or technical assistance. If Step 
One is not completed or if instruments and assistance are not available, 
alternative sources exist. An evaluation guide focusing on program 
objectives can be developed locally. This guide should be kept simple 
and be designed to provide direction to the evaluation team. The guide 
should contain: 

1) A statement of purpose or objective for each specific aspect 
of the program to be reviewed. 

2) Suggested facts and figures relating to each statement. 

3) Selected trends and projections for each statement. 

^) Suggested methods of arriving at conclusions and recommenda- 
tions. 

Other possible sources of instruments include: 

1) Educational Resources Information Center (ERIC). 

2) Research Coordinating Units « 

3) College and University files cf doctoral dissertations and 
master theses or- reports. 

k) Regional Accrediting Agencies. 
5) State Accrediting Agencies. 
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The *1Selected References" contain but a few of the many useful 
instruments and guides that have been developed. The publicacions 
followed by an ED number are generally available through the Educational 
Resources Information Center (ERIC). Requests for the others should be 
made to the listed address. 

STEP EIGHT 
Reporting the Results 



The evaluating team should make an oral report. The audience for 
this oral report could include the teaching and administrative staffs, 
and members of both the governing board and advisory committee. This 
oral report will provide clarification of questionable points. 

A written report should also be sulmitted. At this point, an effec- 
tive writer can be a valuable aide to the evaluation team, putting into 
written form their concltisions and recommendations. The following is a 
suggested format for the report. 

1) Introduction - General description of the evaluation 
activity including: 

a. Items evaluated. 

b. Basic limitations of the evalxiation effort. 

2) Conclusions relating to each item evaluated. 

3) Specific recommendations for improvement or solutions 
to identified problems. 

k) Sunmary of the report and overall assessment. 

5) Appendix - Data collected and other information pertinent 
to the rei)ort but not listed elsewhere. 
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STEP NINE 
Utilizing the Results 



This step shoiild be obvious. After the oral report has been pre- 
sented and copies of the written report have been received and distri- 
buted to appropriate persons^ the reccmmendations should be carefully 
reviewed. Procedtires for implementing the recommendations should be 
developed. Exact implementation of every suggestion may not be practical 
or desirable, but steps should be taken to benefit from the time, efforts, 
and ideas contributed by the evaluation team. Because the team was 
charged to present their ideas for improving the program, they will have 
produced a nvmiber of useful suggestions. 

STEP TEN 
Follow-Up Report to the Evaluation Team 



One month after the completion of the evaluation, a follow-up 
report should be sent to the members of the evaluation team. This 
should be a valuable public relations tool. Most team members will be 
interested in the outcome of their recommendations . A brief report to 
each team member should stimulate his interest in your programs and aid 
in securing his assistance in future activities. A similar report can 
be utilized for release to public information media. 

Suggested items to include in the report follow: 

1) Appreciation for team members* participation. 

2) Report of action planned, underway, and/or completed on 
the recommendations. 
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